an investigation into iranian teachers consistency and bias in evaluation of students writings

while performance-based language assessment has led to an increased authenticity and content validity in the practice of writing assessment, the reliability of ratings has become a major issue. research findings have shown different reactions by native english speaker (nes) and non-native english speaker (nns) teachers to students’ writings. the focus of this study is on investigating whether iranian raters differ from nes raters in their severity when they rate students essays. also, existence of any bias in both groups toward a certain feature of writing is investigated. multi-faceted rasch measurement results showed that iranian raters are significantly more severe than nes raters in rating iranian students writings. also, no significant bias was found in iranian or nes raters toward a certain feature of writing. the results of a survey consisted of likert-type questions also revealed a significant difference between iranian and nes raters in their perceptions of the difficulty of rating organization and grammar. in addition, significant differences were found between the two groups of raters in their perception of the importance of rating content, organization and grammar criteria. the findings implied that rater training sessions should aim at decreasing the overall severity of iranian raters, and deemphasizing the role of grammar while emphasizing the role of inter-sentential discourse features.